Efficient High-Performance Computing with Infiniband Hardware Virtualization

It has been widely accepted that software virtualization has a big negative impact on high-performance computing (HPC) applications performance. This work explores the potential use of Infiniband hardware virtualization in an OpenNebula cloud towards the efficient support of MPI-based workloads. We have implemented, deployed, and tested an Infiniband network on the FermiCloud private IaaS cloud. To avoid software virtualization towards minimizing the virtualization overhead, we employed a technique called Single Root Input/Output Virtualization (SR-IOV). Our solution spanned modifications to the Linux’s Hypervisor as well as the OpenNebula manager. We evaluated the performance of the hardware virtualization on up to 56 virtual machines connected by up to 8 DDR Infiniband network links, with microbenchmarks (latency and bandwidth) as well as with a MPIintensive application (the HPL Linpack benchmark).